Medical imaging system with needle detection capability

ABSTRACT

A system for processing an image data volume acquired with a medical imaging acquisition device  100  from a body comprising a needle is provided. It has been realized it is advantageous to display  150  a plane intersecting the image data volume showing the needle to a user of the system. This allows better positioning of the needle. The system comprises a needle-plane determination  110  module for determining a plane being parallel to and intersecting a representation in the image data volume of the needle and being parallel to a viewing direction. The needle plane determination module may make use of pixel processing and/or spectral transformation, in particular the Gabor transform.

FIELD OF THE INVENTION

The invention relates to a system for processing an image data volumeacquired with a medical imaging acquisition device from a bodycomprising a needle, the system comprising a needle-plane determinationmodule for determining a plane being parallel to and intersecting arepresentation in the image data volume of the needle.

The invention also relates to a medical image acquisition apparatus.

The invention also relates to a method for processing an image datavolume and to a corresponding computer program product and acorresponding system.

BACKGROUND OF THE INVENTION

For some healthcare treatments, patients are injected with a needle totransfer medication or to extract or insert liquids out of and into thehuman body. For some treatments, the accuracy of the injection isimportant. If the needle is guided accurately to a specific place insidea body, e.g. a human or animal body, side-effects of the treatment areminimized. The treatment is typically performed in hospitals by atrained physician or under his supervision.

An example of a treatment where the accurate placement of a needle is aprime consideration is supplying anesthetic to a localized region of abody. The anesthetic may be needed as preparation of surgery or othertreatments or care. Typical targets of the physician or the expert is toinject inside a particular local region, e.g. a particular muscle, nerveor other special element or region of the body.

If the injection needle is positioned by a medical expert, physician orthe like, then the patient relies on the physician's abilities for aprecise positioning of the needle in the considered region of the body.

The expert physician may analyze the local region of the body prior tothe needle injection with a so-called echo-transducer device, which isbased on sending ultrasound frequencies through the human skin into thebody. These sound waves are reflected by the human tissue, nerves, bloodvessels and the like. With a receiving device, the reflections aremeasured, processed and visualized on a display. After reviewing thevisualizations of the local region, the physician performs theinjection. Typically, however, the physician must also base his actionson his expertise and has only minor support from the echo transducerdevice.

SUMMARY OF THE INVENTION

Using an echo-transducer device is clearly useful in deliveringlocal-regional medication such as anesthesia. It turns out however thatusing the echo-transducer device while injecting the needle is not soeasy. Using an echogram a 2D picture is displayed. The thin needle andthe thin ultrasound-field may result in failure to capture the needle.Only with high skill can the physician reliably find the injectionneedle on the displayed echo diagram. Moreover, a physician is neversure that the tip of the needle is being visualized. Part of the needlemay be seen on the echogram, but the tip may be outside the ultrasoundfield.

It is a problem of the prior art that echo-guided loco-regionalanesthesia requires high skill from the physician in operatingdiagnostic sonographic scanners.

The inventors have realized that part of the problem is caused by lackof a precise alignment between the thin needle and the thinultrasound-field. Even if the physician can find part of the needleusing the echo device it is not unlikely that he does not capture thefull needle. The needle may be seen on the echogram, but the tip may beoutside the ultrasound field. If the image data is acquired using animage acquiring head that acquires image data along a two dimensionalplane, we will call that plane the acquisition plane. If the acquisitionplane is visualized, but it is not parallel with the needle, maybediffering as little as a degree, then the tip of the needle may bemissed on the visualization. This is very hard for the user, e.g., thephysician to notice. Especially for procedures wherein precise deliveryof medication is important this is important.

It would therefore be of advantage to a have an improved system forprocessing an image data volume acquired with a medical imagingacquisition device from a body comprising a needle. An improved systemcomprises a needle-plane determination module for determining a planebeing parallel to and intersecting a representation in the image datavolume of the needle, and a viewing direction storage for storing apredetermined viewing direction. The determined needle-plane is parallelto the viewing direction.

Using a medical imaging acquisition device an image data volume may beobtained representing the body wherein at least part of the needle hasbeen inserted. By analyzing the image data using the needledetermination device the needle may be found in the image data. For thephysician this may not be enough. He (or she) should be shown a crosssection which includes the full needle, so that he is sure of seeing thetip of the needle. However, simply selecting any plane coincident withthe needle is confusing for the physician, who may lose his orientation.By aligning the needle plane with a viewing direction this is avoided.At any time the physician is looking at the needle from the same sideand direction. For example, the viewing direction would be orthogonal toa surface of the body in which the needle is inserted and/or the viewingdirection is coplanar with an acquisition plane of the acquisitiondevice. The viewing direction may be a heart-line of an imageacquisition head of the acquisition device.

A physician with the task of guiding the needle into the body further,who is shown a cross section of the image data volume which is parallelto his viewing direction but which contains a representation of theneedle, is sure that he is seeing the tip of the needle. Since theneedle plane is parallel to the viewing direction, the differencebetween this needle plane and the plane the physician is used to lookingat, e.g. the acquisition plane, is minimal, thereby distorting thephysician's sense of orientation to the least extend possible.

Using a high-frequency linear-array transducer with frequencies of 8-17MHz, nerves can be visualized and surrounded by local anesthetics, usinga needle. This procedure results in a higher success-rate, less localanesthetics and is considered to be safer, as it reduced the risk forintravascular injection.

Using an apparatus comprising the system it is easier for the physicianto more accurately perform the needle injection. The safety of thepatient is improved by reducing the chance of an injection at anincorrect location (e.g. in a vein). For the physician it is much easierto use an imaging device, such as an ultrasound system, for thispurpose. The placement of the transducer on the skin above the needle isnot critical anymore and the operator can concentrate completely on theimage and injection itself.

It is noted that the invention can also be used with advantage withimage acquisition device with measure 3d data simultaneously, instead ofmeasuring one 2d acquisition plane at a time. Also in this case, it isuseful for a physician if he sees the plane in a fixed direction, forexample in the same direction in which he himself is looking at thebody.

The image data volume may be a 3D data cube, e.g., represented in acomputer system as a three dimensional array. Not all of the cube needsto be used. For example, the image data volume may also be representedas a data cone, possibly represented in a three dimensional array aswell. Alternatively, polar coordinates may be used in the representationof data.

There are several ways to obtain the desired needle plane. Severaleffective ways of localizing a needle and/or a needle plane in aparticular direction are disclosed in this document. It is noted thatmany of these methods and the devices for executing them haveindependent merit. In particular many of these methods can be used forlocalizing a needle in an image data volume without producing a needleplane in a particular viewing direction. We will discuss solutions tofinding the needle and/or needle plane by processing primarily in thepixel domain and by processing in a transform domain.

For practical purposes, the plane determined to be parallel to andintersecting a representation in the image data volume of the needle maybe regarded as comprising the representation in the image data volume ofthe needle.

In an embodiment, the medical image acquisition apparatus comprises animage data acquisition head configured for acquiring a slice of theimage data volume from the body comprising the needle, the image dataacquisition head being movable with respect to the body in operationaluse. The needle-plane determination module being configured to determinea difference in direction between the needle-plane and the image dataacquisition head, e.g., a difference in direction between theneedle-plane and a direction of the slice acquired by the image dataacquisition head. The system being configured to indicate the differenceto a user of the medical image acquisition apparatus.

As is explained further below, indicating the difference between adirection of the needle plane and the image acquisition head, i.e. adirection of the slice, allows a user of the system to reposition theimage acquisition head so that the slice of the image data volume thatit acquires coincides with the needle plane. After the re-positioningthe user has a better view of the needle since it is better contained inthe plane which the image acquisition head is acquiring. In particular,the risk is reduced that the tip of the needle is not seen, i.e.acquired, by image acquisition head.

In an embodiment, the needle-plane determination module comprises a dataprojection module for projecting the image data volume in a projectingdirection onto a projection plane, and a projected needle-linedetermination module for determining a projected needle-line beingcomprised in the projection plane, the projected needle-line beingparallel to and intersecting a projected representation of the needle inthe projection plane.

By projecting the image data volume, the problem of finding a needle ina three dimensional data volume is reduced to finding a needle in a twodimensional data volume. This simplifies the problem considerably. It isnoted that the plane defined by the projected needle and the projectingdirection comprises the representation of the needle itself in the imagedata volume.

It is generally noted that since the needle is thin it can for practicalpurposes be regarded as a line. Even if the thickness of the needle wereto play a role, the needle may be approximated with its heart line,going through the tip of the needle. If the needle shows up in imagedata as thicker than a line, it can be approximated as a midline.

A line which is parallel to and intersects another line can forpractical purposes be regarded as collinear with the other line.Advantageously, the projected needle-line is collinear with or comprisesthe projected representation of the needle in the projection plane.

In an embodiment, the data projection module is configured forintegrating the data volume along the projecting direction.

The intensity of the needle in the captured image data is found to behigher from the intensity of the body itself. In particular for imagesobtained from sonographic scanners, e.g., ultrasound acquisition deviceproducing an echogram, the needle has been found to produce a higherintensity. By summing the data along the projecting directing thiseffect is amplified. As a result, the position of the needle is found bycapturing the needle as a straight white line in the projected image. Inother words, a two dimensional data histogram is produced from the threedimensional data set.

This can be improved by only considering voxels with an intensity abovea predetermined threshold. A voxel (volumetric pixel) is a volumeelement, representing a value on a regular grid in three dimensionalspace. This reduces the influence of measurement coming from the bodyrelative to measurements coming from the needle. As a result the, needleshows up even better in the projected 2d data set. Automated detectionof the needle can be based on the histogram-forming technique in whichthe intensities, or similar image-forming signal components, areintegrated, e.g. summed within the 3d data set. For example, theintensities may be projected towards the surfaces of a 3D data cube. Foreach (or some) particular point in the projection plane the intensitiesare summed along the line going through the particular point andparallel to the projection direction.

Projecting the data is an example of pixel-based processing.

In an embodiment, the projecting direction is perpendicular to theprojection plane.

In an embodiment, the projecting direction is parallel to the viewingdirection. The needle-plane detector is configured to determine theplane parallel to the viewing direction and comprising the projectedneedle-line. By projecting parallel to the viewing direction the needleplane can be found efficiently. In the projection plane the projectedneedle is found.

For example, using the integrating technique mentioned above, the needlewill stand out. Not that it is sufficient to merely determine aprojected needle line, i.e., a line lying in the projection planecoinciding with the needle. The plane defined by the projectiondirection and the projected needle-line is in this case also parallel tothe viewing direction.

We will refer to a projection plane, orthogonal to the viewingdirection, as the ground plane.

Note that it is not necessary that the projection plane is orthogonal tothe projection direction. This fact may be used for optimizing therepresentation of the projected needle in the projection plane. There isa trade-off involved here, as the projection plane is more orthogonalthe needle, the intensity of the projected needle increases. On theother hand, if the projection plane is parallel to the needle, theprojected needle is longer, thus the projected needle line can bedetermined more accurately.

The projection plane may also be chosen orthogonal to the viewingdirection. This simplifies the implementation and reduces artifactsbased on the fact that the image data need not be of the same thicknessalong all possible projection planes.

The projection embodiment described above have the advantage that it isnot necessary to localize the precise three dimensional location of theneedle. Instead it is sufficient to find a line comprising a projectionof the needle. Any plane going through the projected needle line and theprojection direction also contains the needle. Having a needle plane issufficient for visualization software to show an image of the needle.The physician can see himself where the needle is, and in particularwhere the tip of the needle is.

In an embodiment, the needle-plane determination module comprises aneedle-line determination module for determining a needle-line beingparallel to and intersecting the representation in the image data volumeof the needle. The needle-plane detector is configured to determine theplane parallel to the viewing direction and comprising the needle-line.

It is also possible to determine the three dimensional location of theneedle more exactly. From this information the needle plane can becalculated in a straightforward manner. Note that, also in thisembodiment it is not necessary to find the needle more precisely thanthe needle line, e.g., the tip of the needle need not be explicitlylocated. Having this more precise location of the needle is of coursenot an impediment to using this method of finding the needle plane. Notethat any known needle detection algorithm can be used to determine theneedle line. The needle plane can be determined from the needle line asthe plane comprising the needle line and parallel to the viewingdirection.

In an embodiment, the needle-plane determination module comprises a dataprojection module for projecting the image data volume onto at least afirst projection plane and onto a second projection plane, and a linedetection module for detecting a projected representation of the needlein the projection plane.

By projecting the image data volume onto a projection plane, detectingthe projected needle in the plane a plane can be constructed parallel tothe projection direction and comprising the projected needle. By doingthis twice the needle-line can be found as the intersection of the tweeneedle planes. It may be advantageous to project multiple times to findtwo directions in which the needle can be detected well. Note that thefirst and second projection plane may be orthogonal, but this is notnecessary.

An advantage of determining a needle plane directly from the data or viathe intermediate step of determining a needle line, avoids determiningthe tip of the needle. Finding the precise location of the tip isrelatively hard for image analysis but relative easy for a physicianprovided he tracking the needle on a screen with an acquisition planethat is aligned with the needle plane.

In an embodiment, the needle-plane determination module comprises aspectral transformer for transforming at least part of the image datavolume into a transformation domain using a directionally sensitivespectral transformation.

Apart of determining the needle plane from processing on the pixelsthemselves, the processing may also involve processing in a transformeddomain.

The inventors had the insight that a transform can be chosen whichexplicitly aims at describing the properties of the needle. The keyproperties of the needle are: it is a line segment with a certainorientation. When considering this, two transforms may be used toemphasize those features: a transform that searches for line structures,e.g. the Hough transform, and a directionally-sensitive transform, suchas the Gabor or rotational wavelet transform.

We have evaluated the Hough and Gabor transform for this purpose, and itshowed that the last is superior to the first in detecting the needle.Although, transforming the data slices taken from the 3D data set couldbe done with a signal transform that is particularly sensitive to theoccurrence of line segments, for example, the Hough transform issensitive to line segment, it has been found though that this approachhas its problems. In particular, the Hough transform will react to anyline like material in the image data. When multiple candidates for theline are found the Hough transform gives no good way to select among themultiple possibilities. Instead a better approach is to use adirectionally sensitive transformation.

In an embodiment, the directionally sensitive spectral transformation isa Gabor transform or a rotational wavelet transform. With the Gabortransform, it is possible to determine the sinusoidal frequency andphase content of a specific part of an image. This can be used to detectthe needle.

In an embodiment, the system for processing an image data volumecomprises a data projection module for projecting the image data volumein a projecting direction onto a projection plane, and wherein thespectral transformation is applied to the projection plane.

Using a transform is particularly effective in two dimensions, asadvantage can be taken of the increased intensity of the projectedneedle in the projection plane relative to the other measurements.

In an embodiment, the needle-plane determination module comprises anellipsoid detector for detecting an ellipsoid in the transformationdomain, and a primary axis determination module for determining aprimary axis of a detected ellipsoid.

After transformation of the image data using a directionally sensitivetransform, the needle tends to shows up as an ellipsoid. By detectingthe ellipsoid and determining its primary axis the orientation of theneedle is determined. An orientation in a plane, e.g. of a projectedneedle, may for example be expressed as an angle that a line through theprojected needle makes with a fixed predetermined line, say an axis.

In an embodiment, the needle-plane determination module is adapted totransform the at least part of the image data volume into atransformation domain at least twice wherein the directionally sensitivespectral transformation is configured for different directions, andwherein the ellipsoid detector is configured for selecting a directionfrom the different directions having an ellipsoid in the transformationdomain corresponding to the selected direction having a best fit to theneedle.

A directionally sensitive transform such as the Gabor transform istypically configured to be sensitive to one particular direction. Byperforming the transform for multiple directions a direction can befound in which best shows a particular direction. Since the needleresponds very well to a directional transform, provided the direction inwhich the transform is sensitive corresponds with the direction of theneedle, this direction is presumably the direction of the needle.

In an embodiment, the needle-plane detector is configured to determinethe needle-plane in dependence upon the orientation of the primary axis.

Once the direction of the needle is known it can be found in theprojected image easily. For example, a (virtual) sample needle in thefound direction can be correlated with the projected image data at allpossible locations. The location where the correlation is highestcorresponds to the projection of the real needle.

In an embodiment, the system for processing an image data volumecomprises a display module for displaying a representation of the imagedata in the intersection between the needle plane and the image datavolume.

By showing the intersection of the needle plane to a physician he has abetter understanding of where his needle tip is in relation to theanatomy which may also be seen on the visualization of the intersection.Moreover, if the needle plane is parallel to the physicians viewingdirections, as can be configured in the viewing direction storage, therisk of disorientation on the side of the physician is minimized.

A further aspect of the invention concerns a medical image acquisitionapparatus comprising the system. The visualization is particularlyuseful if it is shown to the physician during the performance of theprocedure. It is noted however that the system can also be employed in amedical image workstation, for processing of image data acquiredearlier. For example, to review an operation done earlier, the methodmay be employed.

In an embodiment of the medical image acquisition apparatus, theapparatus comprises an image data acquisition head configured foracquiring a slice of the image data volume from a body comprising theneedle. The image data acquisition head can be rotated around an axis,the axis being parallel to the viewing direction in operational use. Theneedle-plane determination module is configured to determine adifference in direction between the needle-plane and the image dataacquisition head. The medical image acquisition apparatus is configuredto indicate the difference to a user of the medical image acquisitionapparatus.

For tracking the needle as it is being inserted into the body, whileusing a head which acquires image data along a two dimensionalacquisition plane, it is best if the needle is lying in the acquisitionplane. As noted above however, it is not so easy to align theacquisition plane with the needle. By acquiring a volume of image data,the apparatus can determine the position of the needle with respect tothe head. In particular, a difference in orientation between the needleplane and the acquisition plane can be determined. For example, thedifference in orientation may be expressed as the angle in degrees. Forexample, the needle plane and the acquisition plane intersect aprojection plane, say the ground plane, in two lines, the angle thesetwo lines make can be expressed as an angle in say degrees and shown tothe user. By reporting a similar angle with respect to a differentprojection plane, say the plane parallel to the viewing direction butorthogonal with the needle plane, a second orientation difference can bedetected and reported to the user. In this way the user can be guided toalign his image acquisition head with the needle plane in a particularlyeffective manner.

A further aspect of the invention concerns a method for processing animage data volume acquired from a body comprising a needle with amedical imaging acquisition device. The method comprises storing apredetermined viewing direction, determining a needle-plane being aplane parallel to and intersecting a representation in the image datavolume of the needle. The needle-plane is parallel to the viewingdirection.

A method according to the invention may be implemented on a computer asa computer implemented method, or in dedicated hardware, or in acombination of both. Executable code for a method according to theinvention may be stored on a computer program product. Examples ofcomputer program products include memory devices, optical storagedevices, integrated circuits, servers, online software, etc.

In a preferred embodiment, the computer program comprises computerprogram code means adapted to perform all the steps of a methodaccording to the invention when the computer program is run on acomputer. Preferably, the computer program is embodied on a computerreadable medium.

It will be appreciated by those skilled in the art that two or more ofthe above-mentioned embodiments, implementations, and/or aspects of theinvention may be combined in any way deemed useful.

Modifications and variations of the image acquisition apparatus, of theworkstation, of the system, and/or of the computer program product,which correspond to the described modifications and variations of thesystem, can be carried out by a person skilled in the art on the basisof the present description.

A person skilled in the art will appreciate that the method may beapplied to multidimensional image data, e.g., to 2-dimensional (2-D),3-dimensional (3-D) or 4-dimensional (4-D) images, acquired by variousacquisition modalities such as, but not limited to, standard X-rayImaging, Computed Tomography (CT), Magnetic Resonance Imaging (MRI),Ultrasound (US), Positron Emission Tomography (PET), Single PhotonEmission Computed Tomography (SPECT), and Nuclear Medicine (NM).

A system for processing an image data volume acquired with a medicalimaging acquisition device from a body comprising a needle is provided.It has been realized it is advantageous to display a plane intersectingthe image data volume showing the needle to a user of the system. Thisallows better positioning of the needle. The system comprises aneedle-plane determination module for determining a plane being parallelto and intersecting a representation in the image data volume of theneedle and being parallel to a viewing direction. The needle planedetermination module may make use of pixel processing and/or spectraltransformation, in particular the Gabor transform.

In an improved system for processing an image data volume acquired witha medical imaging acquisition device from a body comprising a needle,the system comprises a viewing direction storage for storing apredetermined viewing direction, and a needle-plane determination modulefor determining a plane parallel to the viewing direction and parallelto and intersecting a representation in the image data volume of theneedle. The needle-plane determination module determined the plane, i.e.the needle plane, in dependency upon the viewing direction.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is explained in further detail by way of example and withreference to the accompanying drawings, wherein:

FIG. 1 is a block diagram illustrating an embodiment of a medicalacquisition device according to the invention,

FIG. 2 is a block diagram illustrating four embodiments of aneedle-plane determination module,

FIG. 3 is a block diagram illustrating a fifth embodiment of aneedle-plane determination module,

FIGS. 4a and 4b illustrates a needle visualization method according tothe invention in the form of a flow chart,

FIG. 5 illustrates a medical acquisition device and workstationaccording to the invention,

FIG. 6 shows an echogram of a body comprising a needle,

FIG. 7 illustrates a data cube with needle and various projectionsurfaces,

FIG. 8 illustrates the projection plane defined by (ax+by+cz+d=0)

FIG. 9 (top) shows a projection of an image volume onto a ground plane,

FIG. 9 (bottom) shows a schematic representation of the projection ofthe image volume onto the ground plane,

FIG. 10a illustrates a transformed domain using the Gabor transform

FIG. 10b illustrates a primary axis,

FIG. 10c illustrates a needle plane,

FIG. 11 shows a physician inserting a needle using an image dataacquisition head,

FIG. 12 shows a misalignment between the needle plane and theacquisition plane.

Throughout the Figures, similar or corresponding features are indicatedby same reference numerals.

LIST OF REFERENCE NUMERALS

-   100 a medical image acquisition apparatus-   110 a needle-plane determination module-   120 a viewing direction storage-   130 a memory-   140 a display module-   150 a display-   160 an image data acquisition head-   210 a data projection module-   220 a projected needle-line determination module-   230 a needle-line determination module-   240 a spectral transformer-   310 an ellipsoid detector-   320 a primary axis determination module-   410 a needle visualization method-   420 obtaining three dimensional image data volume-   425 determining a needle plane-   430 visualizing the needle-   440 a needle visualization method-   450 projecting the image data volume onto a projection plane-   452 transforming the projected image data volume with a Gabor    transform configured for a particular direction-   454 determining the fit of the particular direction-   456 iterating the transforming for a different particular direction-   458 locating the projected needle-   460 determining the needle plane-   500 a needle visualization system-   510 a medical image acquisition apparatus-   520 a medical image acquisition head-   530 a medical image workstation-   540 a needle-   542 tissue-   544 a field of view-   550 a reconstructed image-   600 an echogram-   610 a representation of a needle-   620 needle reflections-   1210 a needle plane-   1220 an acquisition plane-   1215, an angle between a needle plane and an acquisition plane-   1225

Detailed Embodiments

While this invention is susceptible of embodiment in many differentforms, there is shown in the drawings and will herein be described indetail one or more specific embodiments, with the understanding that thepresent disclosure is to be considered as exemplary of the principles ofthe invention and not intended to limit the invention to the specificembodiments shown and described.

FIG. 1 shows a medical image acquisition apparatus 100. Some of thepossible data dependencies between the different elements of medicalimage acquisition apparatus 100 are indicated in the figure with arrows.

Medical image acquisition apparatus 100 is connected, or connectable, toan image data acquisition head 160. Medical image acquisition apparatus100 and image data acquisition head 160 together allow for theacquisition of medical image data from a body, such as a human or animalbody. For example, medical image acquisition apparatus 100 may be anultrasound apparatus and image data acquisition head 160 may beconfigured to send the appropriate sound waves and receive theirreflections.

Image data acquisition head 160 may be a configured for receiving imagedata in three dimensions; in that case the image data volume can beobtained from the acquiring apparatus directly. Image data acquisitionhead 160 may also be an acquisition device that obtains image data alonga two dimensional acquisition plane. In that case a preprocessing stepmay be needed to obtain the image data volume. For example, the operatorof medical image acquisition apparatus 100, e.g., the physician, maysweep image data acquisition head 160 over the surface of the body. Inthis way multiple images are acquired along multiple acquisition planes.Preferably, the multiple acquisition planes are parallel orapproximately so. Preferably, the needle is kept stationary during thesweep. This way of acquiring images is best suited if the body movesrelatively little, for example, for injection in regions away from theheart. Medical image acquisition apparatus 100 may contain an imagevolume preprocessor (not shown) for combining multiple two dimensionalimages obtained from multiple acquisition planes into a single threedimensional data volume. In general, known techniques for combining twodimensional slices into a three dimensional image volume, in particularas related to ultrasound images, can be used with the invention.

Image data acquisition head 160 may comprise an ultrasound transmitter.The ultrasound transmitter may be configured for emitting a beam with acircular opening angle of, say, 60-90 degrees. The reflections measuredin image data acquisition head 160, result in an insight of arectangular slice of the body of a certain depth. The length of theslice is determined by the aforementioned spreading angle of theemitter. By slowly moving the device over the patient's body, thephysician obtains a view of the local region of the body, and seesbasically a series of slices.

Medical image acquisition apparatus 100 comprises a memory 130 forstoring the image data volume. Memory 130 may also be used as temporaryprocessing memory. Medical image acquisition apparatus 100 comprises aviewing direction storage 120. Viewing direction storage 120 may becomprised in memory 130. The viewing direction can be stored in the formof a vector. The viewing direction may also be implicit. For example,medical image acquisition apparatus 100 may be configured that theviewing direction is assumed to be the heart line of image dataacquisition head 160, e.g. a centerline of the acquisition plane.

Medical image acquisition apparatus 100 comprises a needle-planedetermination module for determining the needle plane. The needle planeis a plane intersecting the image data volume, intersecting therepresentation of the needle in the image data volume, and runningparallel to the viewing direction. Different embodiments of needle-planedetermination module 110 are disclosed below. Needle-plane determinationmodule 110 is connected to memory 130 for access to the image datavolume. Needle-plane determination module 110 is connected to viewingdirection storage 120 for access to the viewing direction.

Medical image acquisition apparatus 100 comprises a display module 140and a display 150. Display module 140 is configured for selecting andprocessing the image data volume for displaying on display 150. Displaymodule 140 is connected to memory 130 for access to the image data, andto needle-plane determination module 110 for access to the determinedneedle-plane. Display module 140 may also have access to the viewingdirection. For example, given the image data and the needle-plane,display module 140 may compute a representation of the intersection ofthe image data volume and the needle plane and show it on display 150.It is possible the needle plane does not coincide exactly with thecurrent acquisition plane of image data acquisition head 160. In thatcase display module 140 may show two images, one of the current acquiredimages and one from a computed intersection.

FIGS. 2 and 3, showing embodiments of needle plane determinationmodules, are discussed further below.

FIG. 4a shows a flows chart that illustrates a method for needlevisualization 410 according to the invention. FIG. 4b is a more detailedembodiment which is described further below.

In step 420 three dimensional image data volume is captured. The imagedata volume may be obtained using a device that acquires in threedimensions. The image data volume may also be obtained by combiningmultiple two dimensional images. In step 425 needle plane is determined.There are various ways to obtain the needle plane. For example, theneedle plane can be obtained directly from the image data, using onlypixel based processing or using also spectral transforms. The needleplane can be obtained by first determining a needle line, possibly froma tip and starting point of the needle. From a needle line, a needleplane can be determined. In step 430 the needle is visualized. Forexample, an intersection is shown of the image data volume with theneedle plane. The needle may also be shown in other ways, for example,the needle or needle-line can be shown in a three dimensional renderingof the image data volume.

FIG. 5 shows how a set-up that may be used for needle visualization andin particular to perform the method shown in FIGS. 4a and/or 4 b.

The image data volume may be obtained by a medical image acquisitionhead 520 (also referred to as a probe) connected to a medical imageacquisition apparatus 510. FIG. 5 shows a probe which is capable ofobtaining three dimensional measurements. The probe is acquiring imagesfrom a tissue 542 in which a needle 540 has been partly inserted.

Medical image acquisition head 520 has a limited field of view 544.Probe 520 is connected to medical image acquisition apparatus 510. Theimage data volume can be processed on medical image acquisitionapparatus 510 or on a processing workstation 530. A display shows thereconstructed image 550 including a representation of the needle. Notethat the needle plane is not shown in FIG. 5. A schematic indication ofthe needle plane is shown in FIG. 10c (discussed below).

If needle 540 is redirected, a new needle plane can be recalculatedalmost immediately and displayed. This option is especially useful if athree dimensional acquiring probe is used.

FIG. 6 shows an echogram obtain from a tissue with a needle such atissue 542. Clearly visible is a needle 610. As can be seen in FIG. 6 itis not clear if the tip of the needle is included in the echogram, andif so where exactly it is located. Presumable, the acquisition plane wasnot perfectly aligned with the needle plane. From a collection of imagessuch as shown in FIG. 6 the image volume can be reconstructed. From theconstructed image volume a constructed echogram like picture can becomputed showing the full needle, having the same viewing direction asFIG. 6. Even if the acquisition plane is not perfectly positioned, theuser the system can obtain the correct intersection, showing the needlealong its full length, in particular showing the tip of the needle. FIG.6 also shows needle reflections 620. The needle reflections areartifacts caused by the used ultrasound technique. It is presumed thatthe needle reflections are related to the thinness of the needle. Aswill be further discussed below, the reflections may be a hindrance indetermining the location of the needle and/or needle plane.

FIG. 11 shows a physician performing the procedure on an agar phantom.Shown is a physician inserting a needle 540 into tissue 542 with onehand, while the other holds a medical image acquisition head 520 toguide his actions. FIG. 11 will be discussed more fully below.

FIG. 2 shows embodiments 201, 202, 203 and 204 of needle-planedetermination module 110. FIG. 3 shows embodiment 205.

Needle-plane determination module 202 comprises a needle-linedetermination module 230. In general any needle-line determinationmethod can be used to obtain the needle plane. Below we describe oneparticular way of obtaining the needle-line using projection.

For this purpose needle-plane determination module 202 may comprise dataprojection module 210 and a projected needle tip determination module(not shown).

For the localization of the needle line (also referred to as needleaxis) in a 3D image data one can use parallel projection. A projectiontransformation of the 3D dataset onto a given set of projection planeswith different orientations can be applied. FIG. 7 shows this inschematic form. A data cube having a needle inside is projected onseveral projection planes (four projection planes are shown). If theneedle is perpendicular to the projection planes, it will show up as ahigh intensity peak on the projection plane. The needle tip is found bytaken an optimal threshold that exploits prior probability densities ofthe needle and background voxel intensities. For example, a pixelintensity can be predetermined, a pixel on a projection plane with anintensity above the predetermined pixel intensity is assumed tocorrespond to the needle plane tip. Once the needle tip is found, theneedle line can be easily determined. It is the line which is parallelto the projection direction which passes through the found projectedneedle tip.

This can be formalized as follows:

To determine the position of the needle in 3D US data, the parallelprojection can be used. By applying the parallel projectiontransformation on a 3D dataset for a given set of projection planes withdifferent orientations we map an image as a function I(x, y)representing volume data to plane function P(u, v, α, β). Parameter P(u,v, α, β) describes its projections as a function of the 2-D displacement(u, v) and the projection direction determined by two angles (α, β),formally expressed asP(x,y,α,β)=ΣI(R(α,β)*(u,v)),where R(α, β) is the projection matrix which is depending on thedirection of normal vector of projection plane. The normal vector of theplane is rotated around the x-axis by angle a, and around the y-axis byangle β.

$\overset{arrow}{n} = \langle {1,0,0} \rangle$${S(\alpha)} = {{{\overset{arrow}{n}\begin{bmatrix}{\cos(\alpha)} & {\sin(\alpha)} & 0 \\{- {\sin(\alpha)}} & {\cos(\alpha)} & 0 \\0 & 0 & 1\end{bmatrix}}{R( {\alpha,\beta} )}} = {{S(\alpha)}\begin{bmatrix}{\cos(\beta)} & 0 & {\sin(\beta)} \\0 & 1 & 0 \\{- {\sin(\beta)}} & 0 & {\cos(\beta)}\end{bmatrix}}}$

By summation of all voxel intensities in the same directions as thenormal vector of the projection plane, we project the 3D volume to the2D surface. The result will be a high peak on one of the projectionplanes if the needle is parallel to that normal vector of the plane.Once a projection plane is found in which the needle shows up as singlepoint, the needle line can be obtained as the line which is incidentwith the projected tip and parallel to the projection direction.

To compute the maximum parallel projection transformation we discretethe orientation of our projection planes by the discretization steps Δαand Δβ. Δα and Δβ must be sufficiently fine to avoid missing the needle.

Needle-plane determination module 201 comprises a data projection module210 and a projected needle-line determination module 220.

It is not always required that the needle itself is detected but onlythe vertical plane through it. It is possible to detect thetwo-dimensional plane directly from the 3D dataset, instead of detectingthe one-dimensional needle and then taking the vertical plane throughit. Below one way of accomplishing this is described. Data projectionmodule 210 projects the image data volume onto a projection plane. Wewill use the ground plane in this example embodiment. The ground planeis orthogonal to the viewing direction. We will also assume that theprojection direction and the viewing direction are parallel, althoughthis is not necessary.

The dataset d(x, y, z) is reduced to two dimensions:

${d^{\prime}( {x,y} )} = {\sum\limits_{z}^{\;}{d( {x,y,z} )}}$

for all (x, y). Note that in this representation the z-axis correspondswith the projection direction. By summing all values with identical xand y values, all data points corresponding to one vertical line aresummed. The summation preserves needle information but suppress noise asmuch as possible. In a refinement of this embodiment, a filter can beused in a direction aligned with the needle's orientation. The noise canfurther suppressed by applying a threshold to the data before summating.For example, all data values below a noise threshold are ignored for thesummation. A resulting two dimensional dataset d′(x, y) contains theinformation needed to detect the vertical plane in which the needle ispositioned. The only information that is lost is the location of theneedle inside this vertical plane.

This method was tested on a real datasets. The resulting dataset isshown in FIG. 9 (top). The needle information is clearly visible asdesired. The needle position is schematically illustrated in FIG. 9(bottom). This two-dimensional dataset can now be used to feed a fasttwo-dimensional line detector, e.g., a Hough transform. The needle planecan be determined as the vertical plane going through the detectedprojected needle, that is, the plane through the detected projectedneedle parallel to the viewing direction.

This algorithm works best if the brightness of the needle is highcompared with the brightness of the tissue. That is if the needle is ofsuch material that is reflects ultrasound well and/or is not too thin.The difference in brightness in the datasets can also be caused by thedependency of the needle's brightness on the direction in which thescans are taken. If the contrast between the projected needle and thesurrounding tissue is sufficient a line detector will work. However, ifthe contrast is lower a line detector may not be sensitive enough, eventhough enough information regarding the needle is present in theprojected image. In that case a directionally sensitive transformproduces better result. Even though a full line may not be sufficientlyvisible enough of the direction may is typically still present.

Needle-plane determination module 203 comprises a spectral transformer240. Using a spectral transform is advantageous since it can be arrangedto be sensitive to a particular direction. As a result thetransformation is less sensitive to noise which is typically lessdirectional.

Moreover, a directional transform has the advantage that it is not, orsignificantly less, disturbed by needle reflections which may be presentin the echogram. See, for example, FIG. 6, the reflections indicated at620. The reflections generally run in the same direction as the needleand do not influence a correct determination of the needle direction. Asecond phase, determining the needle location and/or needle plane fromthe found needle orientation, is also less influenced by the reflectionssince the real needle will generally give a better fit than itsreflections.

Before applying spectral transformation the data set may be reduced to atwo dimensional data set, as is illustrated in FIG. 9. This method maybe employed by needle-plane determination module 204 which comprises adata projection module 210 and a spectral transformer 240.

We will discuss a transformation based detection method usingneedle-plane determination module 205 comprises a data projection module210 and a spectral transformer 240, an ellipsoid detector 310 and aprimary axis determination module 320.

The Gabor transform is a special case of the short-time Fouriertransform. First, the image is multiplied by a Gaussian function to givemore weight to the area of interest. Then, the Fourier transform is usedto obtain information about the frequency content of that specific partof the image. The complete definition is stated below (See Jian-Jiun,2007).G _(x)(t,f)=∫_(∞) ^(∞) e ^(−π(τ−t)) ² e ^(−j2πfτ) x(τ)dτ,andG _(x)(t,f)≈∫₂ ² e ^(−π(τ−t)) ² e ^(−j2πfτ) x(τ)dτ.

The Gabor transform makes it possible to search for a specific thicknessand direction of the needle. The thickness is known beforehand. A searchalgorithm is used that starts with a number of candidate angles between0 and 180 degrees, say 4 candidates, and afterwards iteratively zooms inon the best match. This algorithm converges quickly, as it doubles theprecision after each iteration cycle. In only 7 iterations, the angle ofthe needle can be estimated with an accuracy of less than 1 degree. Seethe table below

Precision Iteration (degrees) 1 45 2 22.5 3 11.25 4 5.63 5 2.81 6 1.40 70.70

With filtering, the output of the filter may be correlated to the actualneedle position. For example, the sum of all pixels of a filter outputturns out to be a good criterion for how close the angle of the needleis to the angle at which the filter output gives a maximum response.

For example, a Gabor transform may be carried out on the projectionapplied in FIG. 9. The result is a two dimensional data set, in theGabor domain, which may be visualized. Schematically the visualizedimage looks like FIG. 10a . The Gabor transform can also be combinedwith other ways to reduce a 3 dimensional data volume to a two dimensiondata set.

The high intensities of the needle samples accumulate into anellipsoidal figure with the orientation of the needle in the data cube.This is one of the benefits of the Gabor transform. To determine thisorientation we may use image analysis of the Gabor plane. For example,the image can be de-noised and with simple analysis techniques such asthresholding and segmentation techniques, the ellipsoidal figure isidentified. Similarly, by finding the direction with the maximum amountof high luminance (intensity) samples, the primary orientation is found.The primary axis of the ellipsoid figure is shown in FIG. 10b .

The third step in the analysis is shown in FIG. 10c . Here the primaryaxis of orientation of the ellipsoidal figure is used to construct aplane in the three dimensional data set. This plane contains the needlecompletely, as the orientation was outlined by the Gabor transform.

If desired, the needle can be detected by taking out this plane from thecube. We have then reduced the problem to detecting the needle in thattwo dimensional plane. For example, in this plane, the Gabor transformcan again be applied, or the needle is found directly with other imageanalysis techniques, because the data in this plane was alreadyprocessed in earlier steps.

Below an example is described of the Gabor transform on an image datavolume. One frame of the image data volume is shown in FIG. 6. Theneedle in FIG. 6 has an angle of 103.5 degrees. Using this number as aninput, the following estimates are made of the angle:

Iteration Guess (degrees) Error (degrees) 1 112.5 9 2 101.25 2.25 395.63 7.87 4 104.06 0.56 5 102.66 0.84 6 103.36 0.14 7 103.71 0.21

The algorithm calculates all 7 iterations in about 0.7 seconds usingMatlab code. Although the algorithm makes one poor guess at the thirditeration, it still converges quickly to the true angle. By manuallyrotating the input figure 180 times by one degree the algorithm isfurther tested. It turns out that the algorithm converges to an error ofless than one degree within 7 iterations for all 180 tested angles.

After the iterations an estimate of the angle of the needle are known.This leaves the exact location of the needle still open for detection.With the orientation angle of the needle known, there are a number ofreasonable localization algorithms with which the needle's positionand/or the needle plane can be found. For example, a sample needle withthe estimated angle could be (virtually) inserted and then correlated tothe image data to find the actual needle's position. Another optionwould be to use a data matching algorithm such as block matching inmotion compensated coding, for which many fast algorithms exist. Ourresults in convergence and measuring the orientation of the needleindicate that the Gabor transform is a much better choice than the Houghtransform. Furthermore, fast algorithms exist for the Gabor transformenabling a real-time implementation.

To summarize, we have found that a directional transform compared to aline sensitive transform is less likely to give false positives, i.e.reporting a needle where there is none; faster; and, more robust in thepresence of needle reflections.

FIG. 11 shows a physician performing the procedure on an agar phantom.Shown is a physician inserting a needle 540 into tissue 542 with onehand, while the other holds a medical image acquisition head 520 toguide his actions. An image data volume has previously been obtained bysweeping medical image acquisition head 520 over tissue 542. From theimage data volume a needle plane is determined.

The medical image acquisition device also detects the currentorientation of the acquisition plane in the image data volume. Forexample, the data obtained in the current acquisition plane can becorrelated to the image data volume. The highest correlation correspondsto the location of the acquisition plane.

The image data acquisition head is movable with respect to the body inoperational use. For example, the head may be rotated around an axis,e.g. his centerline or any other axis, or translated along a vector.Typically, a slight translation and/or rotation will be sufficient tocorrect the alignment. The needle-plane determination module may beconfigured to determine a difference in direction between theneedle-plane and the acquisition plane. By indicating the difference toa user of the medical image acquisition apparatus the difference inorientation, the user can correct the positioning of the head. In thisway the acquisition plane is aligned with the needle plane better,allowing better life tracking of the insertion of the needle.

For example, FIG. 12 shows a needle plane 1210 and an acquisition plane1215 schematically, which are not aligned. By intersecting both planeswith a ground plane a difference in orientation can be expressed as anangle; in FIG. 12 as angle 1225. Any projection on a plane allows themeasurement of an angle expressing a difference in orientation. Forexample, by projecting on a plane orthogonal to plane 1210, angle 1215is obtained. By displaying or otherwise reporting angles 1215 and 1220to the user, he can correct the orientation of the image acquisitionhead.

FIG. 4b illustrates with a flow chart a method 440 of visualizing aneedle. In step 420 a three dimensional image data volume is obtained oftissue comprising a needle. In step 450 the data volume is projected inthe viewing direction onto a projection plane, e.g., a ground plane. Insteps 452, 454 and 456 a Gabor transform is applied multiple times fordifferent directions. In step 452 the image data volume projected instep 450 is transformed using the Gabor transform configured for aparticular direction. In step 454 the fit of that particular directionis determined. That is, it is determined how well the Gabor transformresponds to this image. For example, the total intensity of thetransformed image is summed, and taken as a measure of the magnitude ofthe response of the Gabor transform. In step 456 it is determined if theGabor transformation must be repeated for further directions. Forexample, step 456 may control an iteration of a predetermined number ofdirections, i.e., angles. The directions may be chosen uniformly over180 degrees. For example, the transform may be repeated 180 times, oncefor each direction. The direction showing the best fit is kept for lateruse in the method. A more efficient way, is to partition all directionsin a number of sectors, say four quadrants, and perform a Gabortransform for each sector. The sector showing the best fit isiteratively subdivided into smaller sectors for which the process isrepeated. In this way the algorithm zooms into the direction with thebest fit. In step 458 the projected needle or only the projected needleline is determined. This may be accomplished by correlating theprojected image will a number of needle and/or needle lines having thekept direction. The highest found correlation corresponds to thelocation of the needle. In step 460 the needle plane is determined, asthe plane through the needle and/or needle line and parallel to theprojecting direction. The algorithm may proceed with locating theprecise location of the needle in the needle plane, but this is notnecessary. In step 430 the results are visualized, e.g., by showing theintersection of the image data volume with the needle plane.

Note that the method may also be used to determine the needle locationwithout using a viewing direction. In this case the projection may bedone in any suitable direction. The projection may be done for differentdirections if the results are not sufficient, e.g., the projection maybe done for three orthogonal directions.

Methods for processing an image data volume acquired from a bodycomprising a needle with a medical imaging acquisition device can beimplemented in various ways. For example, the method may comprisestoring a predetermined viewing direction, determining a needle-planebeing a plane parallel to and intersecting a representation in the imagedata volume of the needle and parallel to the viewing direction. Themethod may work with without a viewing direction in this case theneedle-plane need not be parallel to the viewing direction. The methodmay also determine the location of the needle and/or needle line.

Many different ways of executing the method are possible, as will beapparent to a person skilled in the art. For example, the order of thesteps can be varied or some steps may be executed in parallel. Moreover,in between steps other method steps may be inserted either. The insertedsteps may represents refinements of the method such as described herein,or may be unrelated to the method. A given step may not have finishedcompletely before a next step is started.

A method according to the invention may be executed using software,which comprises instructions for causing a processor system to performthe method. Software may only include steps taken by a content receivingdevice, or only those taken by an on-demand server, or only those takenby a broadcaster. The software may be stored in a suitable storagemedium, such as a hard disk, a floppy, a memory, etc. The software maybe sent as a signal along a wire, or wireless, or using a data network,e.g., the Internet. The software may be made available for downloadand/or for remote usage on a server.

It will be appreciated that the invention also extends to computerprograms, particularly computer programs on or in a carrier, adapted forputting the invention into practice. The program may be in the form ofsource code, object code, a code intermediate source and object codesuch as partially compiled form, or in any other form suitable for usein the implementation of the method according to the invention. It willalso be appreciated that such a program may have many differentarchitectural designs. For example, a program code implementing thefunctionality of the method or system according to the invention may besubdivided into one or more subroutines. Many different ways todistribute the functionality among these subroutines will be apparent tothe skilled person. The subroutines may be stored together in oneexecutable file to form a self-contained program. Such an executablefile may comprise computer executable instructions, for example,processor instructions and/or interpreter instructions (e.g. Javainterpreter instructions). Alternatively, one or more or all of thesubroutines may be stored in at least one external library file andlinked with a main program either statically or dynamically, e.g. atrun-time. The main program contains at least one call to at least one ofthe subroutines. Also, the subroutines may comprise function calls toeach other. An embodiment relating to a computer program productcomprises computer executable instructions corresponding to each of theprocessing steps of at least one of the methods set forth. Theseinstructions may be subdivided into subroutines and/or be stored in oneor more files that may be linked statically or dynamically. Anotherembodiment relating to a computer program product comprises computerexecutable instructions corresponding to each of the means of at leastone of the systems and/or products set forth. These instructions may besubdivided into subroutines and/or be stored in one or more files thatmay be linked statically or dynamically.

The carrier of a computer program may be any entity or device capable ofcarrying the program. For example, the carrier may include a storagemedium, such as a ROM, for example a CD ROM or a semiconductor ROM, or amagnetic recording medium, for example a floppy disc or hard disk.Furthermore, the carrier may be a transmissible carrier such as anelectrical or optical signal, which may be conveyed via electrical oroptical cable or by radio or other means. When the program is embodiedin such a signal, the carrier may be constituted by such cable or otherdevice or means. Alternatively, the carrier may be an integrated circuitin which the program is embedded, the integrated circuit being adaptedfor performing, or for use in the performance of, the relevant method.

Some of the embodiments according to the invention make use ofprojection of data. Some of the projection techniques which may be usedto implement the invention are summarized below. The embodiment can becombined with different ways of acquiring image data. Different imagedata acquiring methods are described below.

Projection

In general, projections transform a n-dimensional vector space into am-dimensional vector space where m<n. Projection of a 3D object onto a2D surface is done by selecting first the projection surface and thendefining projectors or lines which are passed through each vertex of theobject. The projected vertices are placed where the projectors intersectthe projection surface. The most common (and simplest) projections usedfor viewing 3D scenes use planes for the projection surface and straightlines for projectors. These are called planar geometric projections.

The simplest form of viewing an object is by drawing all its projectededges. In parallel projection (see FIG. 8), we imagine that the eye ofthe observer is at infinity, and the observer views the scene along afixed direction vector <a, b, c>. This time, each point (x0, y0, z0) ofthe scene is projected to the point where the line through (x0, y0, z0)with direction vector <a, b, c> meets a given plane π (a plane withnormal vector <a, b, c>.

In parallel projection, the points on an object are projected to theview plane along parallel lines (projectors). The view plane (projectionplane) ax+by+cz+d=0 (whose normal n=<a, b, c>), is intersected with theprojector drawn from the object point along a fixed vector V=<p, q, r>.All the points on the object are projected to the view plane alongparallel lines. The views formed by parallel projections vary accordingto the angle that the directions of projection make with the projectionplane and it is classified in to subtypes. If the projection lines areperpendicular (normal) to the image plane of projection, V is along thesame direction as normal n of view plane, then that projection is calledthe orthographic projection, with <p, q, r>=<a, b, c>. The projection isoblique when the projection is not perpendicular to the image plane ofprojection, hence <p, q, r>=/=<a,b,c>.

Let ax+by+cz+d=0 be the projection plane, <p, q, r> the direction of theprojection and coordinates (x0, y0, z0) be a point on the object to beprojected. We start at (x0, y0, z0) and travel along the line indirection <p, q, r> until plane ax+by+cz+d=0 is hit.

The parametric equations of the line are:x=x ₀ +pty=y ₀ +qtz=z ₀ +rt

At the some value of parameter t, when the plane equation is satisfied,we are on the projection plane, so thatax+by+cz+d=0a(x ₀ +pt)+b(y ₀ +qt)+c(z ₀ +rt)+d=0a+b+c+t(ap+bq+cr)+d=0

Solving for the unknown parameter value t, gives

$t = {- \lbrack \frac{{ax}_{0} + {by}_{0} + {cz}_{0} + d}{{ap} + {bq} + {cr}} \rbrack}$provided ap+bq+cr=/=0

Substituting this value of t into the previous line equation for x, yand z gives an expression for the projection point (xp, yp, zp):

$x_{p} = {x_{0} - {p\lbrack \frac{{ax}_{0} + {by}_{0} + {cz}_{0} + d}{{ap} + {bq} + {cr}} \rbrack}}$$y_{p} = {y_{0} - {q\lbrack \frac{{ax}_{0} + {by}_{0} + {cz}_{0} + d}{{ap} + {bq} + {cr}} \rbrack}}$$z_{p} = {z_{0} - {r\lbrack \frac{{ax}_{0} + {by}_{0} + {cz}_{0} + d}{{ap} + {bq} + {cr}} \rbrack}}$

With some manipulation, we can write this as a matrix equation:

$\begin{bmatrix}x_{p} & y_{p} & z_{p} & 1\end{bmatrix} = {\begin{bmatrix}x_{0} & y_{0} & z_{0} & 1\end{bmatrix}\begin{bmatrix}m_{11} & m_{12} & m_{13} & 0 \\m_{21} & m_{22} & m_{23} & 0 \\m_{31} & m_{32} & m_{33} & 0 \\m_{41} & m_{42} & m_{43} & 1\end{bmatrix}}$ $\begin{matrix}{m_{11} = {( {{bq} + {cr}} )/( {{ap} + {bq} + {cr}} )}} & {m_{11} = {( {{bq} + {cr}} )/( {{ap} + {bq} + {cr}} )}} \\{m_{12} = {( {- {ap}} )/( {{ap} + {bq} + {cr}} )}} & {m_{12} = {( {- {ap}} )/( {{ap} + {bq} + {cr}} )}} \\{m_{13} = {( {- {ar}} )/( {{ap} + {bq} + {cr}} )}} & {m_{13} = {( {- {ar}} )/( {{ap} + {bq} + {cr}} )}}\end{matrix}$ $\begin{matrix}{m_{11} = {( {{bq} + {cr}} )/( {{ap} + {bq} + {cr}} )}} & {m_{11} = {( {{bq} + {cr}} )/( {{ap} + {bq} + {cr}} )}} \\{m_{12} = {( {- {ap}} )/( {{ap} + {bq} + {cr}} )}} & {m_{12} = {( {- {ap}} )/( {{ap} + {bq} + {cr}} )}} \\{m_{13} = {( {- {ar}} )/( {{ap} + {bq} + {cr}} )}} & {m_{13} = {( {- {ar}} )/( {{ap} + {bq} + {cr}} )}}\end{matrix}$

Our projector <p, q, r> have same direction as normal vector <a, b, c>.The matrix for projections onto the plane will be:

$\frac{1}{( {a^{2} + b^{2} + c^{2}} )}\begin{bmatrix}{b^{2} + c^{2}} & {- {ba}} & {- {ca}} & 0 \\{- {ab}} & {a^{2} + c^{2}} & {- {cb}} & 0 \\{- {ac}} & {- {bc}} & {a^{2} + b^{2}} & 0 \\{- {ad}} & {- {bd}} & {- {cd}} & 1\end{bmatrix}$

For the needle detection, we can use this projection matrix to projectthe 3D dataset onto the 2D surface.

Image Data Acquiring

There are a number of ways to obtain a 3D dataset, which are discussedbriefly in this section.

When using ultrasound, typically, the data is captured in intensityimages since ultrasound does not involve other components than soundwaves. The image data are thus processed in luminance (Y) pictures. Ifanother principle than ultrasound were to be used, the invention may beapplied to color images equally well.

A first way to obtain the information along the third dimension is touse a 3D probe. This probe can obtain a dataset in very short time span.If the data to be measured changes over time (i.e. a patient contractshis heart), then this is the preferred way to obtain a good dataset.

The second method to acquire 3D information is to use a regular 2D probeand make successive scans at a fixed interval, as discussed earlier. Iftime is not a variable and it is possible to make scans at preciseintervals, this method is a good way to generate a batch of slices whichcan be accumulated into a 3D dataset. The data set can be compressed ifrequired, because of its size. This method is, for example, applicablewhen using a tissue phantom for training purposes. The measured data,and detected needle plane provide good feedback to a physician who istraining, e.g. on tissue phantoms, to improve his alignment of an imageacquisition head with a needle.

The third method is carefully sliding a 2D probe over the volume ofinterest maintaining a constant speed while recording the output imagesin a movie. The slices are afterwards extracted from the resultingmovie. This method works well if the scanned volume is sufficientlystable over the time needed to make the sweep, the probe is not tiltedwhile sweeping and the surface is reasonably flat. The advantage overthe previous method is that it can be applied to a real patient. Theacquired data may be somewhat less accurate than the other methods sincea patient always moves a bit and the sweeping speed is never perfectlyconstant.

It should be noted that the above-mentioned embodiments illustraterather than limit the invention, and that those skilled in the art willbe able to design many alternative embodiments without departing fromthe scope of the appended claims. In the claims, any reference signsplaced between parentheses shall not be construed as limiting the claim.Use of the verb “comprise” and its conjugations does not exclude thepresence of elements or steps other than those stated in a claim. Thearticle “a” or “an” preceding an element does not exclude the presenceof a plurality of such elements. The invention may be implemented bymeans of hardware comprising several distinct elements, and by means ofa suitably programmed computer. In the device claim enumerating severalmeans, several of these means may be embodied by one and the same itemof hardware. The mere fact that certain measures are recited in mutuallydifferent dependent claims does not indicate that a combination of thesemeasures cannot be used to advantage.

What is claimed is:
 1. An ultrasound imaging system comprising: an imageacquisition head; a processor configured to: locate a needle tip byprojection transformation onto at least one plane; determine a needleline in an ultrasound image data volume; determine, based on the needleline, a needle plane in the ultrasound image data volume comprising theneedle line; determine a difference between the needle plane and anacquisition plane acquired by the image acquisition head; and generatean indication for display of the difference between the needle plane andthe acquisition plane to a user of the system.
 2. The system of claim 1,wherein the projection transformation comprises applying a directionaltransform to the image data volume to determine the needle plane.
 3. Thesystem of claim 2, wherein the directional transform comprises a Gabortransform.
 4. The system of claim 3, wherein the Gabor transform isapplied in multiple directions.
 5. The system of claim 1, wherein theindication comprises a representation of relative positions of theneedle plane and the acquisition plane.
 6. The system of claim 1,wherein the indicating comprises displaying the difference to the useron a display.
 7. The system of claim 6, wherein the indicating comprisesdisplaying an angle representing the difference between the needle planeand the acquisition plane.
 8. The system of claim 1, wherein theprocessor is further configured to respond to movement of the imageacquisition head until the acquisition plane is aligned with the needleplane and no difference is determined between the acquisition plane andthe needle plane.
 9. The system of claim 1, wherein determining theneedle plane comprises determining an angle of the needle and performinga localization algorithm to determine the needle plane.
 10. The systemof claim 1, wherein the image data acquisition head comprises a 2Dultrasound probe or a 3D ultrasound probe.
 11. The system of claim 1,wherein the processor is configured to determine the needle plane byselecting a plane in the image data volume that includes the needle andis aligned with a viewing direction stored on the system.
 12. The systemof claim 11, wherein the viewing direction is orthogonal to a surface ofa body in which the needle is inserted.
 13. The system of claim 11,wherein the viewing direction comprises a direction in which the user islooking at a body in which the needle is inserted.
 14. The system ofclaim 11, wherein the viewing direction is co-planar with theacquisition plane.
 15. The system of claim 14, wherein the viewingdirection is along a centerline of the acquisition plane.
 16. The systemof claim 1, wherein the processor is further configured to display theneedle plane in relation to the image data volume.
 17. The system ofclaim 15, wherein the viewing direction is predetermined.
 18. The systemof claim 1, wherein the processor is configured to determine the needleplane using image processing.